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Preface 


How  ironic,  that  on  one  hand,  the  government  is  accused  of  waste,  abuses,  duplication  and 
bureaucracy.  Yet  on  the  other  hand,  you  hear  the  cries  of  federal  agencies  having  to  do  more 
with  fewer  resources.  How  do  we  know  if  federal  agencies  are  running  effectively  and 
efficiently?  How  are  they  measuring  their  worth  and  value  to  this  nation?  The  Government 
Performance  and  Results  Act  of  1993  is  designed  to  bring  this  information  to  the  public.  These 
are  questions  that  I  will  address  in  this  paper.  I  chose  this  topic  because  of  the  Acquisition 
Career  Management  and  Resource  Division’s,  SAF/AQXD,  curiosity  on  this  subject  and  its 
potential  for  being  applied  to  the  Acquisition  Professional  Development  Program  and  its 
processes.  I  always  thought  of  performance  measures  as  a  way  of  doing  business,  however,  I 
quickly  learned  that  agencies  struggle  with  the  concept  of  performance  management, 
development  of  performance  metrics,  linking  performance  metrics  to  key  business  processes,  and 
with  the  collection  and  analysis  of  the  data.  Not  surprisingly,  I  struggled  with  these  issues,  as 
well,  while  tying  them  into  the  Acquisition  Professional  Development  Program.  This  paper  is 
intended  to  serve  as  a  guide.  It  will  address  the  purpose  of  Government  Performance  and  Results 
Act  of  1993,  some  of  the  trials  and  tribulations  since  its  enactment  and  its  significance  to  the 
acquisition  workforce.  It  has  given  me  great  pleasure  to  learn  about  this  topic  and  given  me  the 
opportunity  to  find  out  more  about  the  nature  of  our  business. 

I  would  like  to  extend  my  appreciation  to  Lt  Col  Lombardi,  my  research  advisor,  for  his 
guidance  and  assistance.  Additional  appreciation  is  extended  to  Major  Hubert,  SAF/AQXA, 
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Chief  of  Acquisition  Reform  Deployment,  for  providing  invaluable  insight  on  this  topic  and  to 
Mr.  Matthew  Benavides,  SAF/AQXD,  the  Assistant  to  the  Associate  Director  of  Acquisition 
Career  Management,  for  providing  support  on  this  intriguing  topic. 
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Abstract 

Whether  you  call  it  management  by  objective,  balanced  scorecard,  or  total  quality 
management,  the  principles  of  the  Government  Performance  and  Results  Act  of  1993  (GPRA), 
makes  good  business  sense.  However,  GPRA  is  not  just  a  fly  by  night  concept,  it  is  the  law; 
slowly  taking  hold  and  continuing  to  evolve.  The  purpose  of  GPRA  is  to  ascertain  whether  an 
organization  is  operating  efficiently  and  effectively.  Even  though  the  requirements  of  GPRA 
apply  to  federal  agency  heads,  the  Acquisition  Professional  Development  Program  (APDP) 
should  embrace  its  intent.  The  purpose  of  this  research  paper  is  to  discuss  the  concept  of  GPRA, 
develop  and  link  goals  and  performance  measures  to  key  APDP  processes  and  provide 
recommendations  to  ensure  continuity  of  the  program  and  overcome  the  challenges  regarding 
GPRA  requirements. 

Many  organizations  are  struggling  with  the  principles  of  GPRA  and  are  wrestling  with  the 
development,  collection  and  analysis  of  performance  measures.  The  Air  Force  Career 
Management  and  Resources  Division,  SAF/AQXD,  is  taking  a  step  forward  by  expressing 
interest  in  the  topic  to  enhance  their  business  practices.  SAF/AQXD  is  heading  in  the  right 
direction.  They  recognize  the  importance  of  a  strategic  roadmap.  At  the  forefront  of  their 
strategic  planning  is  top  management  support  to  exploit  strategic  opportunities,  like  the  concept 
of  GPRA  to  monitor  the  professional  development  of  the  acquisition  workforce.  This  research 
paper  is  intended  to  ensure  the  effective  management  of  APDP  by  using  the  principles  of  GPRA 
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as  a  framework. 


SAF/AQXD  leaders  should: 

•  Continue  to  support  the  principles  of  GPRA, 

•  Use  the  suggested  goals  and  performance  measures  as  a  starting  point, 

•  Advocate  recurring  strategic  team  meetings  to  refine  and  analyze  SAF/AQXD  s 
strategic  and  performance  plans,  subsequent  measures  and  results, 

•  Construct  an  avenue  to  solicit,  reflect  and  act  upon  customer  (students,  supervisors, 
functional  boards,  consortium  schools)  feedback, 

•  Investigate  the  integrity  of  performance  data,  and 

•  Maintain  the  aggressive  schedule  to  integrate  data  systems. 

The  data  for  this  paper  was  compiled  through  research  via  the  Air  University  Library, 
Internet  sites,  SAF/AQXA,  the  Acquisition  Reform  Office,  the  Air  Force  Acquisition  Training 
Office  and  the  Air  Force  Manpower  and  Innovation  Agency.  A  limited  number  of  interviews 
were  conducted  to  receive  background  information,  guidance,  and  to  determine  the  extent  of  its 
use  within  acquisition. 


Part  1 


Introduction 

The  intent  of  the  Government  Performance  and  Results  Act  (GPRA)  of  1993  is  to  make 
federal  agencies  accountable  to  the  American  people  for  their  program  performance  and  the 
results  thereof.  It  requires  them  to  set  goals,  measure  their  performance  and  report  their 
accomplishments.  Many  organizations  are  struggling  with  the  principles  of  GPRA  and  are 
wrestling  with  the  development,  collection  and  analysis  of  performance  measures. 

The  acquisition  workforce  must  be  prepared  to  face  the  challenges  of  the  21st  century.  To 
face  these  challenges,  they  must  ensure  that  the  acquisition  workforce  is  well  trained,  flexible 
and  more  effective  than  ever  before.  Agency  heads  for  Agriculture,  Commerce,  Defense, 
Education,  Energy,  etc.,  are  federally  mandated  to  comply  with  GPRA,  however,  these  principles 
can  be  extended  to  all  subordinate  organizations.  The  Air  Force  Career  Management  and 
Resources  Division,  SAF/AQXD,  is  embracing  the  principles  of  GPRA  to  improve  the  primary 
business  of  the  organization — effective  management  of  APDP. 

The  material  herein  is  intended  as  a  guide.  It  addresses  the  intent  of  the  law,  some  of  the 
trials  and  tribulations  since  its  enactment,  and  its  significance  to  the  acquisition  community.  To 
assist  SAF/AQXD  with  the  concept  of  GPRA,  organizational  goals  and  performance  measure 
have  been  developed  and  are  provided  as  a  starting  point.  In  addition,  recommendations  are 
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provided  to  ensure  successful  continuity  of  the  program  and  to  overcome  the  challenges 
regarding  the  requirements  of  GPRA. 
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Part  2 


Background 


The  law  simply  requires  that  we  chart  a  course  for  every  endeavor  that  we  take 
the  people ’s  money  for,  see  how  well  we  are  progressing,  tell  the  public  how  we 
are  doing,  stop  the  things  that  don ’t  work,  and  never  stop  improving  the  things 
that  we  think  are  worth  investing  in. 


— President  Clinton  upon  signing  GPRA 


Definition  of  GPRA 

Public  Law  103-62,  also  know  as  the  Government  Performance  and  Results  Act  of  1993 
(GPRA),  was  passed  to  restore  the  confidence  of  the  American  people  in  the  federal  government. 
This  Act  was  initiated  to  improve  the  efficiency  and  effectiveness  of  federal  programs.  It  is  a 
“performance-based  budgeting”  system  that  attempts  to  measure  performance  and  link  resources 
to  results.  Simplistically,  the  federal  agencies  are  tasked  to  set  goals,  measure  results,  and  assess 
their  impacts. 

Origin  and  its  Purpose 

Republican  Senator  William  Roth  introduced  the  bill  in  the  Senate  Committee  on 
Governmental  Affairs  in  1990.  In  the  two  years  that  followed,  many  hearings  and  modifications 
were  made.  Then  came  a  lull  in  activity  around  the  1992  presidential  election.  Surprisingly,  by 
the  time  the  House  version  of  the  legislation  was  introduced  in  February  of  1993,  the  bill  sailed 
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through  with  overwhelming  support.  President  William  Clinton  signed  the  legislation  on  3 
August  1993.  Its  purpose  is1: 

•  To  improve  the  confidence  of  the  people  in  the  government  by  holding  agencies 
accountable  for  achieving  program  results; 

•  To  stimulate  reform  with  a  series  of  pilot  projects  that  could  be  used  as  example  for 
others; 

•  To  help  managers  improve  service  delivery  by  requiring  them  to  plan  for  meeting 
program  objectives  and  providing  them  with  information  about  program  results, 

•  To  improve  congressional  decision  making  by  providing  information  on  achieving 
statutory  objectives  and  relative  effectiveness  of  various  programs. 


The  ultimate  goal  of  this  latest  performance  budgeting  initiative  is  to  improve  budget  and 
resource  allocation  decisions  by  linking  resources  to  results.  Another  way  of  viewing  this  is  to 
relate  data  on  program  performance  to  appropriation  structures. 


Requirements  of  GPRA 

The  requirements  of  GPRA  are  built  around  three  key  processes:  the  strategic  plan,  annual 
performance  plans  and  annual  program  performance  reports. 

The  GPRA  requires  the  strategic  plan  to  include  : 

•  A  comprehensive  mission  statement  covering  major  functions  and  operations  of  the 
agency. 

•  General,  outcome  related  goals  for  each  major  function  and  operation  of  the  agency. 

•  A  description  of  how  each  of  the  goals  will  be  achieved.  Included  in  the  description 
would  be  the  operational  processes,  skills  and  technology,  and  the  human  capital, 
information  and  other  resources  required  to  meet  the  goals  and  objectives. 

•  Any  external  factors  that  may  affect  achievement  of  goals. 

The  GPRA  requires  the  annual  performance  plan  to  include  : 

•  Objective,  quantifiable,  and  measurable  performance  goals. 

•  A  description  of  the  operational  processes,  skills  and  technology  and  the  human,  capital, 
information,  or  other  resources  required  to  meet  the  performance  goals. 

•  Performance  indicators  to  measure  success  in  meeting  outputs,  service  levels,  and 
outcomes  for  each  program. 

•  A  basis  for  comparing  actual  program  results  with  the  established  performance  goals. 
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The  GPRA  requires  the  annual  performance  report  to  include4: 

•  A  review  of  the  successes  in  achieving  the  performance  goals. 

•  An  explanation  when  a  performance  goal  has  not  been  met. 

•  A  description  of  corrective  actions,  as  required. 

GPRA’s  delivery  schedule  of  these  products  is  outlined  in  this  Four-Step  Plan5: 

1 .  Requires  all  federal  agencies  to  submit  a  five-year  “strategic  plan”  that  contains  general  goals 
and  objectives  for  all  major  functions  and  operations.  The  law  requires  agencies  to  update 
their  strategic  plans  every  three  years.  (Accomplished  September  30, 1997) 

2.  Agencies  develop  an  annual  performance  plan  that  expresses  agency  goals  in  a  measurable 
form  or  alternatively,  that  includes  descriptive  statements  of  minimally  acceptable  and 
successful  programs.  (Submitted  September  1997  for  fiscal  year  99  to  Office  of 
Management  and  Budget  (OMB)  and  has  occurred  every  fiscal  year  thereafter) 

3.  Agencies  deliver  annual  performance  reports  to  the  President  and  the  Congress.  Performance 
reports  are  agency  report  cards  designed  to  measure  the  success  of  programs  by  comparing 
actual  results  with  performance  goals.  (For  the  first  time,  agencies  must  report  on  their 
performance  by  March  3 1 , 2000) 

4.  To  crosswalk  budgets  and  performance. 

As  simplistic  as  these  steps  may  appear,  in  implementing  the  principles  of  GPRA,  many 
challenges  have  been  exposed  since  its  inception,  seven  years  ago.  Organizations  continue  to 
struggle  with  the  principles  of  GPRA.  They  lack  support  and  guidance,  wrestle  with  the 
development,  collection  and  analysis  of  performance  measures  and  are  suspicious  of  its  intent 
and  impact  to  their  programs. 


Notes 

1  Beryl  A.  Radin,  “The  Government  Performance  and  Results  Act  (GPRA):  Hydra-headed 
Monster  or  Flexible  Management  Tool?”,  Public  Administration  Review  58,  no.  4  (July  1998): 
307-316. 

2  Alice  Maroni,  “DoD  implementation  of  the  Government  Performance  and  Results  Act 
(GPRA),”  Armed  Forces  Comptroller  41,  no.  4  (Fall  1996):  23-26. 

3  Ibid. 

4  Ibid. 

5  Francois  Melese,  “The  Latest  in  Performance  Budgeting:  The  Government  Performance 
and  Results  Act  ”  Armed  Forces  Comptroller,  44,  no.  2  (Spring  1999):  19-22. 
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Part  3 


Challenges  In  Implementing  GPRA 

There  is  a  wide  divergence  of  views  about  this  management  reform  effort.  It  is  a  tangled  set 
of  expectations  and  experiences  that  reflect  quite  often-competing  views  about  the  process.  The 
General  Accounting  Office  (GAO)  adamantly  supports  compliance  with  the  GPRA  and  related 
laws.  It  is  seen  as  a  way  to  vigorously  attack  the  problems  of  program  duplication,  waste  and 
mismanagement  within  the  federal  government.  However,  the  following  concerns  challenge 
agencies  in  implementing  this  initiative. 

Agencies  are  slow  to  adopt  performance  based  budgeting  and  are  discouraged  with  this 
effort  for  many  reasons.  They  lack  support  and  guidance,  wrestle  with  the  development, 
collection  and  analysis  of  performance  measures  and  are  suspicious  of  its  intent  and  impact  to 
their  programs. 

Support  and  Guidance 

“Overall  agencies  have  complained  that  OMB  does  not  do  enough  to  support  them,  and  in 
particular  that  it  does  not  do  enough  to  answer  all  their  calls  and  questions.  OMB’s  response  is 
that  it  supports  GPRA,  that  it  is  doing  what  it  can  to  advance  it  within  existing  resources  and 
priorities,  and  that  it  would  be  better  to  grow  performance-based  management  than  to  declare  by 
directive.”1  Congressional  leaders  advocate  OMB  taking  on  a  stronger  leadership  role  in  helping 
agencies  produce  useful  strategic  and  performance  plans.  However,  guidance  and  direction  are 
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minimal.  Agencies  continue  to  struggle  in  developing  performance  based  management  plans  and 
measurements  on  their  own. 

Ironically,  Congress  could  be  the  real  problem.  Congress’  inattention,  critical  attitude  and 
irresponsibility  can  stifle  this  initiative.  They  “must  have  the  political  courage  to  acknowledge 
their  role  in  creating  and  failing  to  solve  the  problems  for  which  they  now  excoriate  agencies. 

In  August  and  November  1997,  House  Majority  Leader  Dick  Armey  graded  federal  agency 
strategic  draft  and  final  plans,  respectively.  The  Department  of  Defense,  along  with  several 
agencies,  received  an  F  on  their  strategic  plans.  As  a  result  of  these  findings,  in  August  1998, 
Congressional  decision-makers  considered  these  strategic  plans  useless.  GAO  responded  by 
saying  legislators  should  shy  away  from  grading  the  process  and  focus  their  attention  on 
redundant  programs  and  resolving  political  choices  brought  to  light  by  the  merits  of  the  program. 
This  program  is  still  considered  in  its  infancy.  It  continues  to  be  a  learning  process  for  all. 
Congress,  OMB,  and  other  agencies  should  build  on  their  experiences  by  working  together  on 
key  performance  issues,  which  will  help  make  future  strategic  plans  more  useful.  In  March 
2000,  Congress  will  have  the  opportunity  to  act  responsibly  with  the  information  agencies  will 
provide  in  their  performance  plans.  Based  on  agency  performance  assessment  input.  Congress  at 
this  point,  can  oversee,  react  and  respond  to  the  analysis  provided  by  the  agencies.  Congress 
should  be  objective,  provide  guidance  and  alert  agencies  of  areas  that  contain  government 
duplication,  weaknesses,  strengths  or  practices  that  inhibit  effective  or  efficient  operations. 

In  addition,  there  is  great  concern  and  debate  in  government  circles  today  regarding  mid¬ 
manager  involvement  and  training  in  the  “performance”  movement.  “Christopher  Wye,  Director 
of  the  Center  for  Improving  Government  Performance  at  the  National  Academy  of  Public 
Administration,  said  in  a  Sept  6  (1999)  commentary  in  Federal  Times  that  most  employees  active 
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in  the  performance  movement  are  motivated  largely  out  of  personal  initiative  and  enthusiasm, 
not  by  agency  design."4  On  the  other  hand,  no  matter  how  promising  an  idea,  other  sources 
indicate  few  mid-managers  will  take  up  the  lead  without  top  management  support.  Some  mid¬ 
managers  state  performance  goal  setting  is  the  top  manager’s  responsibility  and  top  managers 
should  seek  their  input,  which  is  seldom  done.  In  October  1999,  Sen  George  Voinovich,  R-Ohio, 
requested  another  survey  from  GAO  to  determine  middle  management  involvement  in 
performance  based  programs.  GAO  hopes  the  survey  will  indicate  how  seriously  the  Results  Act 
is  being  taken  in  all  tiers  of  the  federal  government. 

Data  Collection 

“The  greatest  hurdle  we  face  to  getting  good  performance  reports  is  the  quality  of  data  the 
government  produces.”5  There  is  great  skepticism  regarding  the  data  received  considering  many 
agencies  lack  or  have  outdated  systems  that  do  not  track  output.  A  GAO  audit  of  federal 
financial  statements  revealed  that  the  government’s  ability  to  conduct  its  affairs  is  severely 
limited  due  to  financial  weaknesses,  issues  in  fundamental  record  keeping,  incomplete 
documentation,  and  weak  internal  controls.6  “GPRA  not  only  requires  reasoning  out  the  raison 
d’etre  for  existing  programs  and  the  desired  results  for  new  ones,  it  also  means  measuring 
outcomes.  But  measurement  rests  on  good  data,  a  commodity  many  say  is  sadly  lacking  in  most 
agencies  today.”7  Additionally,  agencies  face  challenges  in  evaluating  program  results  and 
ensuring  the  validity  and  integrity  of  the  data. 8  As  a  result  of  issues  regarding  the  validity  and 
integrity  of  this  data,  Congress  and  OMB  must  be  sensitive  to  and  judicious  with  the  analysis 
received. 
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Uncertainty 


“The  path  to  performance-based  management  is  fraught  with  uncertainty  about  many  crucial 
factors,  including  the  extent  of  leadership  support,  the  intended  uses  of  the  performance  data,  the 
long-term  availability  to  support  performance  data  systems  and  the  validity  and  reliability  of  the 
data  themselves.”9  Also,  management  feels  GPRA  will  either  cut  their  program  or  reduce  their 
funding  allocation,  especially  on  poorly  performing  or  highly  visible  political  programs. 
Because  of  these  fears,  the  information  being  reported  by  the  managers  may  not  be  a  true 
indication  of  the  health  of  the  organization.  To  make  this  initiative  a  success,  it  will  require 
support,  receptive  organizational  cultures  and  incentives.  Support  must  come  from  agency 
leadership  influencing  the  cultural  change  and  institutionalizing  incentive  programs  to  show  the 
benefits  of  having  an  efficient  and  effective  organization. 


Notes 

1  Christopher  Wye,  “Results  Act  is  Getting  Results  from  Bottom  Up,”  Federal  Times  35,  no. 
31  (6  Sep  1999):  15. 

2  Anne  Laurent,  “The  Results  Act:  Playing  Chicken,”  Government  Executive  30,  no.  1, 
(January  1998):  51. 

3  Brian  Friel,  “GAO  Suggests  Performance  Plan  Fixes,”  n.p.;  on-line,  Internet,  January  2000, 
available  from  http://www.govexec.com/dailyfed/0998/090998bl.htm 

4Editorial,  “Focus  on  Performance  Should  Start  at  the  Top,”  Federal  Times  35,  no.  41  (15 
November  99):  30. 

5  Fred  Thompson,  “Faulty  Data  Plagues  Compliance  with  Results  Act,”  Federal  Times  35, 
no.  33,  (20  September  1999):  21. 

6  ibid 

7  Anne  Laurent,  “Performance  Anxiety,”  Government  Executive  30,  no.  3  (March  1998):  20- 

26. 

8  Ibid. 

9  Kathryn  Newcomer,  “Results-Oriented  Management  Requires  Culture  Change,”  Federal 
Times  35,  no.  6,  (7  June  1999):  15. 
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Part  4 


Why  Apply  GPRA  to  APDP 


Who  hangs  on,  wins . 


— German  Proverb 


To  begin  the  discussion  of  applying  GPRA  principles  to  the  Acquisition  Professional 
Development  Program  (APDP),  let’s  look  closely  at  who  SAF/AQXD  is,  what  their 
responsibilities  encompass,  and  then  elaborate  as  to  why  they  should  apply  these  principles. 


Who  is  SAF/AQXD? 

The  Career  Management  and  Resources  Division  resides  in  Rosslyn  VA  with  a  satellite  Air 

Force  Acquisition  Training  Office  at  Randolph  AFB  TX  servicing  military  and  civilian  members 

worldwide.  Its  primary  responsibility  is  to  ensure: 

Air  Force  compliance  with  the  requirements  of  Defense  Acquisition  Workforce 
Improvement  Act  and  promote  professional  development  of  the  acquisition 
workforce.  SAF/AQXD  is  responsible  for  acquisition  workforce  determination, 
career  management  policy  and  oversight,  support  of  the  Director  of  Acquisition 
Career  Management  and  the  Acquisition  Professional  Development  Council, 
acquisition  career  management  liaison  with  OSD  and  sister  services,  acquisition 
training,  and  acquisition  functional  representation  to  AF  personnel. 

The  above  resides  under  the  auspices  of  the  Acquisition  Professional  Development  Program 
(APDP). 
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What  is  APDP? 


The  Acquisition  Professional  Development  Program  (APDP)  is  an  Air  Force 
career  development  program  for  military  and  civilian  personnel  occupying 
acquisition  positions.  It  was  established  by  the  Assistant  Secretary  of  the  Air 
Force  (Acquisition)  in  response  to  the  1989  Defense  Management  Report  (DMR) 
and  subsequently  the  Defense  Acquisition  Workforce  Improvement  Act 
(DAWIA)  of  1990.  DoD  5000.52-M,  "Defense  Acquisition  Education,  Training 
and  Career  Development  Program,"  Nov  95  established  specific  standards  to  be 
met  by  persons  occupying  acquisition  positions.  To  ensure  individuals  meet  these 
standards,  the  Air  Force  established  the  APDP  through  which  individuals  can  be 
certified  at  Level  I  (Basic),  Level  II  (Intermediate),  and  Level  III  (Advanced).  The 
certification  checklists  provide  a  concise  description  of  the  education,  experience, 
and  training  required  to  meet  the  standards  for  certification  in  acquisition  career 
fields.2 


What  is  DAWIA? 

DAWIA"  refers  to  the  1990  Defense  Acquisition  Workforce  Improvement  Act.  Its 
purpose  is  to  improve  the  effectiveness  of  the  military  and  civilian  acquisition 
workforce  and  thereby  improve  the  acquisition  process.  The  legislation  is  specific 
on  the  minimum  qualification  standards  of  those  performing  functions  integral  to 
the  acquisition  process,  and  defines  critical  acquisition  positions. 

By  recognizing  these  terms,  it  will  be  easier  to  understand  the  designing  and  development  of 
the  performance  measurement  system. 


Why  GPRA? 

Changes  have  been  inevitable  and  SAF/AQXD  must  continue  to  prepare  and  face  the 
challenges  of  the  21st  century.  In  preparation  for  the  future,  they  must  measure  how  well  they 
are  doing.  Performance  measures  are  a  living  document.  They  should  be  adjusted  and  revised  as 
the  need  arises  to  meet  the  objectives.  By  monitoring  performance  measures,  SAF/AQXD  can 
have  a  profound  effect  not  only  in  efficiency  and  effectiveness,  but  also  in  quality,  service  and 
cohesiveness.  The  results  will  guide  them  toward  achieving  the  mission.  The  results  rule.  The 
toughest  challenge  is  making  clear  the  connection  between  programs  and  results.  The  program 
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needs  to  be  designed  to  protect  their  investment,  the  acquisition  workforce.  Why  protect  this 
investment?  The  stakeholders  demand  it.  Who  are  the  stakeholders?  Besides  the  American 
public,  it  is  the  Warfighter.  In  using  the  framework  of  GPRA,  a  well  trained  workforce  to  get  a 
product  or  service  “faster,  better  and  cheaper”  to  the  Warfighter  is  put  in  place. 

By  applying  the  principals  of  GPRA  it  should  improve  management  decision  making, 
promote  better  internal  management  of  programs  and  increase  accountability  to  everyone. 
Effective  planning  will  enhance  mission  performance  by  defining  priorities  and  identifying 
specific  areas  management  should  pay  close  attention  to.  There  is  a  “need  to  recognize  and 
measure  what  is  important,  what  makes  sense,  things  that  display  trends  over  time  and  things  that 
tell  us  what  needs  to  be  done  and  done  better.”4  In  addition,  understanding  GPRA  may  soon  be 
critical  to  obtaining  funding  for  programs  within  the  Department  of  Defense.  Let’s  look  at  some 
techniques  that  will  help  in  designing  an  effective  performance  measurement  system. 


Notes 

1  Draft  Acquisition  Career  Management  Functional  Activities,  no  date. 

2  “Air  Force  Acquisition  Training  Office  Operating  Procedures  for  Acquisition  Training, 

October  1999,  1-22;  on-line,  Internet,  January  2000  available  from 

http://www.safaq.hq.af.mil/acq_workf/training/policy.html 

3  “The  Acquisition  Professional  Development  Guide,”  n.p.;  on-line,  Internet,  January  2000, 
available  from  http://www.safaq.hq.af.mil/acq_workf/apdp/apdp.html 

4  “Performance  Measures  to  Support  Mission  Essential  Tasks,”  n.p.;  on-line,  Internet, 
January  2000,  available  from 

http://www.afmia.randolph.af.mi1/afmia/mip/mipp/perf_mgt/documents/dir_of_perf_mngt_doc.h 

tm 
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Part  5 


Successful  Performance  Based  Measurement  Techniques 


Many  things  difficult  to  design  prove  easy  to  performance. 

— Samuel  Johnson 

In  the  past  ten  years,  the  acquisition  environment  has  seen  many  technological  and 
legislative  changes  that  have  impacted  their  processes  and  acquisition  workforce.  Even  with  vast 
changes  and  scarce  resources,  the  acquisition  workforce  is  expected  to  deliver  timely  best  value 
products  or  services  to  the  customer,  while  upholding  public  policy  and  trust.  To  achieve  this 
effort,  the  public  has  entrusted  SAF/AQXD  under  DAWIA  to  groom  and  “professionalize”  the 
acquisition  workforce.  How  do  they  define  and  shape  the  acquisition  workforce  of  today  and 
tomorrow,  take  steps  to  improve  its  effectiveness,  and  partner  with  policy  makers  to  ensure  that 
the  APDP  policy  provides  the  framework  to  achieve  this  vision?  One  such  tool  is  to  embrace  the 
principles  of  GPRA.  Agency  heads  are  federally  mandated  to  comply  with  GPRA,  however,  the 
principles  can  extended  to  all  subordinate  organizations.  Outlined  below  are  steps  SAF/AQXD 
should  consider  when  designing  a  performance  based  measurement  system: 

Designing  a  Performance  Based  Measurement  System 

The  design  principles  for  an  effective  performance  measurement  system  are  as  follows1 : 

1 .  To  formulate  a  clear,  coherent  mission,  strategy  and  objectives. 

•  Objectives  flow  from  strategy,  providing  the  basis  for  development  of  specific 
measures  and  standards.  Include  secondary  and  intangible  objectives. 
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•  Revise  or  redefine  periodically  as  the  program  evolves. 

2.  To  develop  an  explicit  measurement  strategy. 

•  Use  the  formal  program  mission,  strategy  and  objectives  as  a  foundation. 

•  It  is  the  blueprint  for  the  design  and  development  of  the  measurement  system. 

•  Measurement  system  includes  the  categories  of  measures  called  for  (input,  output, 
outcome,  etc);  data  definitions,  data  collection,  storage,  access,  and  reporting 
formats;  and  specification  of  the  technological  environment. 

•  Provide  for  the  needs  of  all  important  users 

3.  To  involve  key  users  in  the  design  and  development  phase. 

•  To  acquaint  key  decisionmakers  and  other  interested  parties  with  the  power  and 
limitations  of  the  measurement  system,  their  active  participation  in  the  design  and 
development  phase  is  well  advised.  Mere  consultation  is  not  sufficient.  Although 
this  is  time-consuming  and  arduous  process,  it  is  necessary  in  order  to  lend 
credibility  to  the  effort  as  well  as  to  avoid  future  misunderstandings. 

4.  To  rationalize  the  programmatic  structure  as  a  prelude  to  measurement. 

•  Take  account  of  the  crucial  relationship  between  program  structure  and 
meaningful  measurement. 

•  Review  multi-program/multi-goal  to  disentangle  diverse  programs  where  no 
compelling  argument  can  be  offered  for  their  continued  aggregation. 

•  Extensive  reorganization  of  programs  (and  agencies)  is  not  out  of  the  question. 

5.  To  develop  multiple  sets  of  measures  for  multiple  users,  as  necessary. 

6.  To  consider  the  customer(s)  of  programs  and  systems  throughout  the  process. 

•  Consider  the  needs,  desires  and  satisfaction  of  customer  at  each  stage  of  the 
organizational  learning  process:  formulation  of  goals  and  objectives,  development 
of  measures,  and  evaluation  for  organizational  learning  and  quality  improvement. 

•  It  is  especially  important  to  relate  customer-satisfaction  measures  to  operational 
measures  so  the  impact  of  changes  in  the  efficiency  and  quality  of  operational 
results  can  be  examined  in  the  context  of  customer  needs  and  satisfaction  levels. 

7.  To  provide  each  user  with  sufficient  detail  for  a  clear  picture  of  performance. 

8.  To  periodically  review  and  revise  the  measurement  system. 

•  Engineer  flexibility  into  the  design  of  the  performance  measurement  channel. 

9.  To  take  account  of  upstream,  downstream,  and  lateral  complexities. 

10.  To  avoid  excessive  aggregation  of  information. 

As  mentioned  earlier,  development  of  performance  management  system  will  not  come 
easily. 

“Decision  makers  must  understand,  in  some  intimate  sense,  the  nature  of  the 
systems  which  they  seek  to  monitor,  in  all  of  their  inherent  complexity.  The 
following  concerns  will  be  inherent  in  the  design,  development  and  operation  of 
virtually  any  performance  measurement  system: 

•  Divergent  perspectives  (different  audiences  require  different  information), 

•  Unclear  mission  and  objectives  (a  fact  of  governmental  life), 

•  Multiple  and  contradictory  organizational,  program,  and  system  goals, 
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•  Monitoring  vs.  evaluation  informational  needs  (Bamow  1992), 

•  Lack  of  consideration  of  the  full  range  of  outputs  and  outcomes  (Osborne  and  Goebler, 
1992),  and 

•  Measuring  customer  satisfaction  in  a  regulatory  environment  (Swiss,  1992) 

Performance  measurements,  at  best,  can  be  a  useful  supplement  to  a  closer,  more 
in-depth  understanding  of  programs  and  the  systems  in  which  they  are  embedded. 

But  they  cannot  replace  the  expert  judgments,  which  are  borne  of  direct 
experience.  Such  expertise  is  the  true  basis  for  learning,  adaptation,  and 
improvement.”2 

“Attributes  of  a  good  performance  measures  are  meaningful  (reinforce  goals/requirements), 
understandable  (logical,  communicated),  reliable,  valid,  show  trends  over  time,  economically 
collected,  timely,  and  drives  appropriate  action-improvement.”3  The  critical  piece  of  this  act  is 
developing  successful  measures  and  analyzing  their  results.  Therefore,  “successful  measures 
must  be  relevant  (applicable,  pertinent,  and  important),  specific  (defined,  distinct,  and  explicit), 
measurable  (assessable  and  can  be  calculated),  agreed  upon  (settled  and  resolved)  and 
communicated  (announced  and  made  known).”4  Receiving  timely,  reliable  and  standardized 
information  will  reveal  where  resource  decisions  have  to  be  made.  Applying  proper  analysis 
from  accurate  and  reliable  source  data  will  ultimately  reduce  costs  to  the  organization. 

Developing  measures  should  also  begin  by  benchmarking  against  like  organizations,  but 
SAF/AQXD’s  sister  service  counterparts  have  not  begun  the  process  of  formally  linking 
performance  to  results.  Therefore,  they  are  clearly  paving  the  path. 

Getting  reliable  performance  data  and  motivating  managers  to  use  it  will  become  more 
important  each  day.  The  concept  of  GPRA  has  taken  hold  within  the  Air  Force.  The  Air  Force 
has  termed  it  “Performance  Management.”  For  further  direction,  guidelines  for  explaining  the 
processes  and  tasks  associated  with  “operationalizing”  quality  have  been  published  in  AF 
Instruction  90-1102,  Performance  Management.  This  instruction  explains  how  performance 
management  principles  will  improve  the  Air  Force  organization  and  how  mission  essential  tasks 
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are  accomplished.  Simplistically,  the  Air  Force  performance  management  steps  are  “plan,  do 
and  assess.”  Naturally,  these  steps  coincide  with  the  principles  of  GPRA. 

Department  of  Defense  Measurement  Systems 

As  we  know,  GPRA  requires  all  federal  agencies  to  engage  in  a  cycle  of  strategic  planning, 
performance  planning,  and  performance  evaluation  report.  In  step  with  the  requirements  of 
the  law,  the  Department  of  Defense  submitted  their  strategic  plan,  the  May  1997  Report  of  the 
Quadrennial  Defense  Review  (QDR),  to  Congress  and  OMB  by  September  of  1997.  Shortly 
thereafter,  the  Department  adopted  a  mission  statement  and  corporate  level  goals  reflecting  the 
QDR.  The  performance  plan  was  submitted  to  Congress  with  the  budget  for  FY99  (Appendix  J 
of  the  Annual  Defense  Review).  The  performance  evaluation  report  is  to  be  provided  to 
Congress  by  March  2000. 

Appendix  J  of  the  Annual  Defense  Review  is  DoD’s  GPRA  performance  plan.  Within  this 
appendix,  parts  of  GOALS  3  and  6  refer  to  the  acquisition  workforce  and  other  acquisition 
elements.  Under  GOAL  3,  “prepare  now  for  an  uncertain  future  by  pursuing  a  focused 
modernization  effort  that  maintains  U.S.  qualitative  superiority  in  key  war-fighting 
capabilities,”5  the  acquisition  workforce  is  addressed  in  detail  under  Appendix  F,  Defense 
Acquisition  Workforce  Implementation  Report.  Appendix  F  displays  the  Defense  Acquisition 
Workforce  Improvement  Act  (DAWIA)  reporting  requirements  for  all  the  components.  Some  of 
the  other  elements  addressed  under  GOAL  6,  “fundamentally  reengineer  the  Department  and 
achieve  a  21st  century  infrastructure  by  reducing  costs  while  maintaining  required  military 
capabilities  across  all  DoD  mission  areas”6  are  to: 

•  Create  a  world  class  learning  organization  by  offering  40  or  more  hours  annually  of 
continuing  education  and  training  to  the  DoD  acquisition-related  work  force  and 
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•  Eliminate  layers  of  management  through  streamlined  processes  while  reducing  the  DoD 
acquisition-related  work  force  by  15  percent. 

These  issues  fall  under  the  umbrella  of  SAF/AQXD.  SAF/AQXD  mission  statement,  goals 
and  measures  are  explored  to  ensure  congruency  with  the  requirements  of  the  DoD  performance 
plan  (the  Annual  Defense  Review)  and  to  show  the  criticality  of  pursuing  the  development  and 
employment  of  performance  measures. 


Notes 

1  Robert  S.  Kravchuk  and  Ronald  W.  Schack,  “Designing  Effective  Performance- 
Measurement  Systems  Under  the  Government  Performance  and  Results  Act  of  1993,”  Public 
Administration  Review  56,  no.  4  (July  1996):  348-358. 

2  ibid 

^  “Performance  Measures  to  Support  Mission  Essential  Tasks”,  n.p.:  on-line,  Internet, 
January  2000,  available  from 

http://www.afrnia.randolph.af.mil/afmia/mip/mipp/perf_mgt/docurnents/PMtmg.ppt 

4  ibid 

5  “Appendix  J,  GPRA  Performance  Plan  for  FY1999”,  n.p.:  on-line,  Internet,  13  January 
2000,  available  from  http://www.dtic.mil/execsec/adr98/apdx  i.html 

6  ibid 
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Part  6 

Applying  GPRA  Agenda  to  APDP 

If  you  don ’t  know  where  you  are  going,  you  will  probably  end  up  somewhere  else. 

— Laurence  J  Peter 

The  GPRA  requires  the  strategic  plan  includes  a  comprehensive  mission  statement, 
outcome-related  goals  and  an  annual  performance  plan  to  ensure  results  meet  the  goals. 
SAF/AQ  mission  statement  and  goals  will  be  reviewed  and  used  as  a  reference  to  develop, 
describe  and  support  the  proposed  goals  and  performance  measures  for  SAF/AQXD. 


SAF/AQ  Mission  Statement  and  Goals 

SAF/AQ’s  mission  statement  is  clear,  concise  and  flexible: 

Provide  the  leadership,  direction,  policy  and  resources  to  acquire  superior 
systems,  supplies,  and  services  to  accomplish  the  Air  Force  mission. 

SAF/AQXD’s  mission  statement  supports  AQ  in  the  following  manner 

Develop  and  implement  human  resources  policy  and  oversee  the  effective 
use,  training  and  management  of  organic  and  support  contractor  personnel 
supporting  the  Air  Force  acquisition  mission.  Direct  compliance  with  the 
Defense  Acquisition  Workforce  Improvement  Act. 

SAF/AQXD  focuses  on  the  acquisition  personnel  needed  to  support  the  overall  mission 
while  satisfying  the  requirements  of  DAWIA.  As  we  can  see,  the  mission  statements  are 
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congruent,  a  critical  step  of  GPRA.  The  next  step  is  to  develop  goals  and  objectives  that 
describe  how  we  are  going  to  accomplish  the  mission. 

Excerpts  of  SAF/AQ’s  goals  and  objectives  that  are  applicable  to  the  APDP  process  are 
provided: 

OUR  PEOPLE1: 

Goal  1:  Invest  in  our  people  for  the  future. 

We  support  our  people.  We  will  continually  and  aggressively  strive  to  improve 
communication,  training  programs,  and  activities,  which  will  foster  esprit  de 
corps. 

Objectives: 

•  Keep  our  people  informed  and  solicit  candid  feedback. 

•  Commit  to  train,  develop,  and  manage  our  people  to  maintain  a  professional 
team. 

•  Strengthen  emphasis  on  career  development. 

The  above  goal  advocates  the  professional  development  of  acquisition  personnel. 

INTERNAL  PROCESS 

Goal  2:  Optimize  SAF/AQ  Internal  Process 

We  constantly  work  to  improve  our  productivity.  Increase  productivity  will  allow 
us  to  more  efficiently  meet  our  customer’s  needs  while  at  the  same  time  lead  the 
reengineering  of  the  acquisition  process  to  deliver  weapon  systems  better,  faster, 
and  cheaper. 

Objectives: 

•  Conduct  reengineering  of  internal  SAF/AQ  processes 

•  Implement  an  integrated  acquisition  information  system. 

The  above  goal  allows  all  of  AQ  to  make  significant  strides  and  improvements  to  the  way 
we  do  business  and  operate. 
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ACQUISITION  SYSTEM  EFFECTIVENESS 
Goal  3:  Improve  Acquisition  System  Effectiveness 

As  the  policy  maker  for  Air  Force  Acquisition,  SAF/AQ  directly  impacts  the 
methods  used  to  acquire  DOD  weapon  systems.  We  must  strive  to  improve  Air 
Force  acquisition  to  deliver  weapons  better,  faster,  and  cheaper. 

Objectives: 

•  Aggressively  streamline  the  acquisition  process  to  improve  responsiveness. 

•  Improve  acquisition  planning  to  ensure  program  success. 

•  Ensure  maximum  use  and  dissemination  of  best  practices  to  improve  process 
efficiency. 

•  Provide  technology  options  to  the  warfighters  to  improve  combat  capability 
and  reduce  the  cost  of  ownership. 

The  above  goal  focuses  on  ensuring  we  have  the  processes  in  place  to  shape  and  prepare  the 
acquisition  workforce  (with  knowledge,  skills  and  ability)  to  respond  to  the  warfighter  needs. 

EXTERNAL  RELATIONSHIPS 

Goal  4:  Build  and  Improve  External  Relationships 

To  be  credible  and  effective  participant  in  the  process  to  procure  weapon  systems 
that  meet  mission  needs,  SAF/AQ  must  build  and  maintain  solid  relationships 
with  the  MAJCOMs,  OSD,  Congress  and  industry.  We  must  work  diligently  to 
provide  innovative  solutions  to  user  challenges  to  meet  mission  needs  as 
affordably  and  rapidly  as  possible. 

Objectives: 

•  Improve  overall  working  relationships  with  OSD  and  Congress  to  ensure 
credibility  and  improve  our  products. 

The  above  goal  emphasizes  sharing  information  with  customers  to  improve  our  business 
process. 

CUSTOMER  SATISFACTION 
Goal  5:  Satisfy  our  Customer’s  Needs 
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Customer  Satisfaction  is  the  final  test  of  our  performance  as  a  team.  The  only 
method  to  ensure  that  we  are  meeting  our  mission  requirements  is  to  ensure  that 
we  are  satisfying  our  customer’s  requirements. 

Objectives: 

•  Identify  our  customers  and  how  we  interact  with  them 

•  Improve  product/service  quality 

The  above  goal  stresses  satisfying  customer  needs.  The  information  received  from 
customers  is  critical  in  developing  measures  of  success. 

At  the  time  of  this  paper,  the  goals  for  SAF/AQXD  were  still  a  work  in  progress.  They  had 
not  yet  been  released  and  were  unavailable  for  this  report.  However,  SAF/AQXD  has  a  detailed 
strategic  roadmap  in  place  that  describes  their  focus  on  the  future,  desired  end  state  and  action 
plans  on  how  to  get  there.  They  are  focusing  on  five  major  thrust  areas:  (1)  the  Acquisition 
Workforce  2002  and  beyond,  (2)  Acquisition  Resource  Management,  (3)  DAWIA  and  the  future, 
(4)  Total  Workforce  Management  and  (5)  Cultivating  Legislative  Understanding  and  Support. 
Even  though  SAF/AQXD  is  working  on  this  step  of  the  strategic  planning  process,  the  following 
goals  and  performance  measures  are  proposed  as  a  guide  and  a  contribution  based  on  the  above 
indicators. 


Proposed  Goals 

SAF/AQXD ’s  charter  is  to  enhance  the  quality  of  the  acquisition  workforce  by  attracting 
qualified  personnel,  improving  the  skills  and  knowledge  of  current  personnel,  and  maintaining 
the  very  highest  professional  standards  while  keeping  abreast  with  policy  changes.  Besides 
supporting  AQ’s  goals,  the  proposed  AQXD  goals  pay  close  attention  to  the  organization’s 
required  activities.  Unfortunately,  the  proposed  goals  will  have  to  be  redressed  by  the 
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organization  to  ensure  they  cover  varied  expectations.  The  proposed  goals  fall  short  of  involving 
key  users,  an  important  design  principle  for  an  effective  measurement  system.  Active 
participation  of  key  decision  makers/customers  is  encouraged  to  lend  credibility  to  the  process 
and  ensure  customer  desires  are  met.  With  this  in  mind,  the  proposed  goals  are  provided  as  a 
baseline. 

Proposal  -  SAF/AOXD  Goal  1:  Comply  with  and  respond  to  DAWIA  and  any  other 
legislative  mandates,  professional  development  initiatives,  and  changes.  The  reason  this  goal 
is  important  is  because  SAF/AQXD  will  continue  to  have  responsibility  of  adhering  to  the 
provisions  of  DAWIA  and  any  other  changes  which  will  impact  or  influence  the  makeup  of  the 
acquisition  workforce.  Therefore,  we  must  be  cognizant  of  our  workforce  structure  and 
impending  gaps.  This  goal  affords  SAF/AQXD  the  opportunity  to  improve  and  question  the 
provisions  of  DAWIA  and  identify  potential  approaches  to  benefit  the  readiness  of  the 
acquisition  workforce. 

Proposal  -  SAF/AOXD  Goal  2:  Train  and  sustain  qualified  members  of  the  acquisition 
corps.  This  goal  is  important  because  SAF/AQXD  must  monitor  the  total  acquisition  workforce 
structure  to  determine  that  it  meets  the  Air  Force’s  operational  and  modernization  plans  of  today 
and  the  future.  Qualification  and  certification  standards  in  accordance  with  DAWIA  must  be 
met.  We  must  ensure  our  acquisition  workforce  is  optimally  prepared  to  face  the  challenges  of 
tomorrow. 

Proposal  -  SAF/AOXD  Goal  3:  Improve  the  acquisition  training  cycle.  The  training  life 
cycle  process  begins  with  the  identification  of  the  mandatory  acquisition  training  requirements 
from  the  field  to  the  posting  of  successful  completion  in  the  student’s  personnel  record. 
Presently,  the  training  cycle  is  complex,  laborious  and  riddled  with  various  management 
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systems.  By  monitoring  the  throughput,  avenues  to  innovate  and  compress  the  training  cycle 
process  for  faster,  better,  and  cheaper  results  will  be  identified. 

Proposal  of  SAF  AOXD  Goal  4:  Bestow  a  quality  Acquisition  Professional 
Development  Program  to  all  our  customers.  This  goal  is  important  as  well  because  a  cohesive 
program  will  produce  the  optimal  product,  a  synergistic  acquisition  workforce.  A  win-win 
situation  will  result  when  we  listen  closely  to  our  customers:  the  public,  Congress,  OSD,  the 
acquisition  workforce,  and  the  warfighter.  SAF/AQXD  must  institutionalize  surveys  to  receive 
feedback  regarding  the  program  and  its  processes.  Are  we  satisfying  the  customer’s  needs,  are 
we  applying  and  employing  the  right  career  progressive  decisions?  Are  we  providing  the  most 
current,  accurate  and  relevant  mandatory  acquisition  training?  Are  we  exploiting  suggestions 
from  our  customers?  We  must  reflect  on  the  feedback  from  the  surveys  and  take  immediate 
appropriate  action. 

Proposed  Performance  Measures 

As  mentioned  earlier,  we  must  measure  what  is  important  to  APDP.  The  proposed 
performance  measures  should  be  considered  critical  to  SAF/AQXD’s  mission  and  their  strategic 
roadmap.  They  are  not  all-inclusive  or  definitive,  but  a  starting  point  in  meeting  the  objectives 
of  GPRA.  A  team  (consisting  of  internal  and  external  customers)  should  validate  the  proposed 
performance  measures,  develop  any  other  measures  and  establish  target  levels  applicable  to  the 
acquisition  career  development  program.  By  consistently  monitoring  these  measures, 
SAF/AQXD  will  know  where  they  have  been,  where  they  are  and  where  they  want  to  be;  the 
importance  of  assessing  performance  measures.  Keep  in  mind  that  one  of  the  keys  to 
performance  measures  is  not  always  quantification  or  numbers,  but  assessment  and  evaluation  of 
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the  information.  The  following  proposed  performance  measures  are  associated  with  the 
proposed  goals: 

Proposal  -  SAF/AOXD  Goal  1:  Comply  with  and  respond  to  DAWIA  and  any  other 
legislative  mandates,  professional  development  initiatives,  and  changes. 

Proposal  -  Metric  la:  Required  vs.  Actual  AQXD  Staffing  Levels 

•  By  Civilian/Military/Other 

•  By  Grade/Rank/Other 

•  By  Series/ AFSC 

•  By  Program 

•  By  Budget  and  Actual  Costs 

Purpose:  To  identify  the  staffing  posture  required  for  supporting  the  execution  of  APDP.  It 
will  also  determine  if  the  operating  budget  can  maintain  the  manpower  structure  as  required  and 
as  it  exists. 

Proposal  -  Metric  lb:  Required  vs.  Actual  Total  Force  Structure  (Acquisition 
Positions) 

•  By  Organic  and  Inorganic  (Contractor  or  Other  Support) 

•  By  Civilian/Military/Other 

•  By  Functional  Area 

•  By  Grade/Rank/other 

•  By  Series/AFSC 

•  By  MAJCOM/Base 

Purpose:  To  identify  the  total  acquisition  workforce  structure  composition  by  the  various 
definitions  of  the  acquisition  corps.  This  information  is  required  to  baseline  the  workforce  by 
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mix  and  levels  and  to  determine  gaps  between  what  is  needed  and  what  we  have.  This  becomes 
crucial  information  for  determining  appropriate  recruitment  actions  and  for  making  end  strength 
adjustments  within  the  appropriate  workforce  stalls. 

Proposal  -  Metric  lc;  Number  of  Qualified  Acquisition  Personnel  Occupying  Non- 
Acquisition  Positions  (same  as  breakdown  lb) 

•  By  name,  location  and  last  time  occupied  an  acquisition  position 

•  Certification  level  and  number  of  years  in  acquisition 

Purpose:  Identify  potential  and  existing  resources  in  the  total  workforce  inventory  that  do 
not  presently  occupy  an  acquisition  position.  Personnel  occupying  non-acquisition  positions 
with  acquisition  credentials  are  a  potential  resource  for  recruitment  purposes.  These  are  viable 
sources  that  we  could  potentially  tap  for  filling  critical  vacancies. 

Proposal  -  Metric  Id:  Status  of  acquisition  career  development  initiatives  to 
complement  the  workforce. 

•  Interchangeability  of  civilian  and  military  positions 

•  Blending  civilian  and  military  professional  career  paths 

•  Acquisition  Demonstration  Project  (Selective  field  testing  of  an  alternative  civilian 
personnel  system,  performance  based,  to  streamline  and  provide  a  method  of  progression 
through  the  ranks.  Employee  compensation  is  based  on  contribution  to  the  mission.) 

Purpose:  These  programs  are  being  introduced  to  permit  flexibility  and  consistency  in  the 
personnel  and  professional  development  process.  Close  monitoring  of  the  effects  of  the  above 
programs  can  help  managers  overcome  delays  in  processing  personnel  actions  or  ensuring  the 
same  career  development  opportunities  to  all  acquisition  personnel. 
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Proposal  -  SAF/AQXD  Goal  2:  Train  and  sustain  qualified  members  of  the  acquisition 


corps. 

Proposal  -  Metric  2a;  Certification  Levels  of  the  Acquisition 

•  Same  breakdown  as  Metric  lb 

Proposal  -  Metric  2b:  Number  of  Certification  Waivers  Issued  (breakdown  like  lb). 

•  Reasons  for  Issuance 

Proposal  -  Metric  2c:  Number  of  acquisition  personnel  not  certified  within  the 
statutory  requirement  of  18  months  (breakdown  like  lb). 

•  Due  to  Education 

•  Due  to  Training  (identify  courses  lacking) 

•  Due  to  Experience  (identify  time  lacking) 

Proposal  -  Metric  2d:  Dollars  Expended  on  Education 

•  Type  of  education  (college,  continuous  education,  etc) 

•  Number  of  students  trained  by  type  of  education 

•  Unfunded  education  requirements 

Proposal  -  Metric  2e:  Dollars  Expended  on  Mandatory  Acquisition  Training 

•  Number  and  Cost  of  Out  of  cycle  requests 

•  Number  and  Cost  of  TDY  by  type  of  course  delivery  mode  (resident,  on-site,  satellite, 
web-based,  etc)  and  number  of  students  trained 

•  Cost  per  student  by  type  of  course  delivery 

•  Unfunded  training  requirements 

Purpose:  All  of  the  above  measures  will  monitor  the  career  progression  of  the  acquisition 
workforce.  It  will  identify  whether  certification  levels  have  been  met  and  education  and  training 
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opportunities  have  been  exploited.  In  accordance  with  the  law,  acquisition  personnel  must  be 
certified  to  the  level  of  their  position  within  18  months  of  occupying  the  position.  The 
individuals  should  meet  education,  training  and  experience  requirements  to  obtain  certification. 
The  above  indicators  maintain  the  integrity  and  inventory  of  qualified  and  certified  professionals 
and  highlight  deficiencies  requiring  corrective  actions. 

Proposal  -  Metric  2f:  Track  Continuous  Education  for  all  acquisition  personnel 
(breakdown  like  lb). 

•  Type  of  training  (i.e.  conference,  lunch  and  learn,  symposium,  etc.) 

•  Number  of  individuals  trained  and  cost  expended  by  organization  and  SAF/AQXD 

•  Number  and  by  name  those  not  completing  80  points  (classroom  hours)  in  two  years 
Purpose:  Presently,  funding  for  continuous  education  is  not  available  from  Congress  even 
though  continuous  education  requirements  have  been  imposed  on  the  acquisition  workforce  to 
maintain  acquisition  proficiency.  Tracking  and  monitoring  of  continuous  education  will  identify 
the  type  of  education  received,  identify  those  who  are  not  in  compliance  with  the  minimum  hours 
required  and  identify  the  costs  being  borne. 

Proposal  -  SAF/AQXD  Goal  3;  Improve  the  acquisition  training  cycle 

Proposal  -  Metric  3a:  Training  requirement  vs.  allocations  vs.  input  vs.  attendance 
(breakdown  like  lb). 

•  By  priority 

•  By  course  delivery  mode 

Proposal  -  Metric  3b:  Attrition  rate  (breakdown  like  lb) 

•  By  reason 

Proposal  -  Metric  3c:  Cancellation  rate  (breakdown  like  lb) 
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•  By  Course/Class/Location 

•  By  Reason 

Purpose:  All  of  the  above  metrics  will  reflect  the  training  requirements  of  our  customers 
and  highlight  the  actual  training  quotas  received,  applications  processed  and  number  of  students 
attending  and  completing  the  courses.  Service  training  seats  are  allocated  based  on  previous  year 
execution  rates.  The  above  metrics  will  baseline  and  identify  mandatory  training  availability  and 
execution  and  underscore  administrative  or  programmatic  changes,  which  validates  or  impacts 
the  training  requirement  by  customer. 

Proposal  of  SAF  AOXD  Goal  4:  Bestow  a  quality  Acquisition  Professional 
Development  Program  to  all  our  customers. 

Proposal  -  Metric  4a:  Demographics  of  the  Acquisition  Workforce  -Service 
Computation  Dates,  Aging,  Retirement  Eligible,  and  Accessions  (breakdown  like  lb) 

Purpose:  To  anticipate  and  adequately  plan  for  reshaping,  “rightsizing”,  and  an  aging 
workforce.  This  information  is  critical  in  posturing  expertise  in  areas  that  will  be  impacted  by 
attrition  factors. 

Proposal  -  Metric  4b:  Solicit,  track  and  address  feedback  surveys  from  our  customers: 
student,  supervisor,  consortium  school,  field  training  managers,  etc. 

•  Take  action  if  suggestion  or  pattern  is  noted. 

Purpose:  To  determine  adequacy  of  the  acquisition  training  and  education  structure  and 
curriculum  by  receiving  feedback  from  customers  and  providing  input  to  schoolhouses, 
instructors  and  functional  boards  to  make  applicable  changes  to  training  material.  The  survey 
will  serve,  as  a  tool,  to  ensure  the  training  is  qualitatively  useful. 
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Proposal  -  Metric  4c:  Identify  training  and  education  cost  savings  alternatives  or 
efforts. 

•  Status  and  Schedule  of  any  systems  integration  or  development 

•  Modernization  of  Personnel  Systems 

•  Defense  Acquisition  University  Operating  Support  System 

•  Modification  of  mandatory  acquisition  training  schedule  to  support  our  requirements 
which  results  in  significant  cost  savings  or  cost  avoidance 

Purpose:  All  of  the  above  measures  recognize  opportunities  to  update  or  upgrade  our 
processes  to  make  them  efficient  or  effective.  If  revisions  are  necessary,  these  changes  should  be 
brought  immediately  to  the  attention  of  appropriate  parties  (consortium  schools,  functional 
boards  and  legislative  committees)  for  resolution. 

Linking  Performance  Measures  to  Fiscal  Accountability 

“Sen.  Ted  Stevens,  R-Alaska  warned  agencies  failing  to  develop  acceptable  strategic  plans 
required  by  the  Results  Act  may  have  their  appropriations  put  on  hold  until  they  comply.” 
Under  the  GPRA,  agencies  are  to  account  for  the  money  they  receive  and  execute,  demonstrate 
the  importance  of  their  program  and  provide  purpose  and  contribution  to  bettering  this  country  to 
the  public.  The  agencies’  efforts  should  concentrate  not  only  on  tracking  spending,  but 
managing  their  finances.  The  APDP  budget  is  approximately  $10M.  Besides,  receiving  this 
direct  appropriation,  SAF/AQXD  is  responsible  for  managing  another  $10M  received  from  the 
Defense  Acquisition  University  to  cover  TDY  expenses  incurred  by  students  attending 
mandatory  acquisition  training  courses  at  consortium  schools.  The  proposed  metrics  should 
measure  the  dollars  provided  to  sustain  the  operation  and  ensure  the  minimal  or  reasonable  level 
of  funding  required  to  monitor  whether  the  business  activities  are  operating  effectively  and 
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efficiently.  Developing  solid  baseline  information  and  collecting  useful  and  accurate  data  to 
measure  performance  at  all  levels  of  the  organization  will  be  the  biggest  challenge  for 
SAF/AQXD. 

Performance  Data  Systems 

The  acquisition  career  management  program  is  comprised  of  a  myriad  of  complex,  multi-use 
knowledge  information  domains,  which  include  legacy  systems  and  new  applications,  hosted  on 
various  platforms.  These  systems  are  either  dated,  cannot  communicate  with  other  systems,  or 
do  not  provide  results  oriented  data.  Overall,  similar  to  other  agencies,  the  management  systems 
are  inadequate  and  not  wholly  linked,  henceforth,  rarely  being  consistent  or  timely.  The 
acquisition  career  management  program  comprises  of  the  following  systems: 

Defense  Civilian  Personnel  Data  System,  Military  Personnel  Data  Systems,  Acquisition 
Career  Management  System,  Expert  Level  Personnel  Review  Tool,  Virtual  Blue  Room,  Defense 
Acquisition  University  Operating  Support  System,  Army  Training  Requirements  and  Resources 
System,  Air  Force  Materiel  Command  Education  and  Training  Management  System,  and  the  Air 
Force  Training  Management  System. 

SAF/AQXD  is  making  strides  in  performance  based  management  by  solving  technology 
problems.  They  acknowledge  and  are  confronting  the  lack  of  good  performance  data  and  related 
systems.  They  are  aggressively  pursuing  these  issues  with  serious  effort.  Currently, 
SAF/AQXD  is  committed  to  creating  an  Integrated  Digital  Environment  for  all  Air  Force 
acquisition  and  sustainment  programs,  which  will  support  all  AQ  business  activities  and  provide 
immediate  access  to  needed  information.  While  integrating  data  systems,  SAF/AQXD  should 
also  take  positive  steps  to  improve  the  accuracy  of  the  information  residing  in  these  systems. 
Because  of  the  nature  of  the  business,  this  will  take  time. 
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Notes 


1  “SAF/AQ,  Assistant  Secretary  of  the  Air  Force  Acquisition,  Strategic  Plan  1996,”  n.p.;  on¬ 
line,  Internet,  January  2000,  available  from  http://www.safaq.hq.af.mil/ acq  ref/strategic 

2  Anonymous,  “Results  Act:  The  Long  Hot  Summer,”  Government  Executive  29,  no.  8, 
(Aug  1997):  6. 
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Part  7 


Summary 


If  we  perform  our  assigned  mission  tasks  with  excellence  and  improve  that 
performance  in  a  measurable  way ,  we  are  operationalizing  quality. 

— General  Michael  Ryan  and  F.  Whitten  Peters 
AF  Chief  of  Staff  and  Acting  Secretary  of  the  Air  Force 

Recommendations 

Even  as  recently  as  July  13,  1999,  at  a  Washington  Conference  called  Excellence  in 
Government”,  agency  officials,  government  experts  and  congressional  officials  addressed 
concerns  surrounding  the  move  toward  a  performance-focused  government.  These  concerns 
were1: 

•  The  OMB  was  not  taking  enough  of  a  leadership  role  in  offering  guidance  to  agencies  as 
they  craft  their  performance  plans 

•  Neither  Congress  nor  OMB  has  issued  criteria  that  will  be  used  to  judge  agency 
performance  reports 

•  The  task  of  gathering  and  disseminating  information  required  to  compile  the 
performance  reports  may  overwhelm  staff  and  information  systems  at  many  offices  and 
agencies 

•  Some  agencies  may  require  unique  measurement  criteria  as  they  assess  their 
performance. 

•  Programs  with  high  political  value  may  need  to  be  reassessed 

•  How  the  performance  of  one  agency  program  will  be  judged  when  other  government 
programs  also  are  aimed  at  the  same  goal.  An  example  is  teen  pregnancy  addressed  by 
both  the  Education  Department  and  the  Health  and  Human  Services  Department. 
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These  concerns  should  not  be  a  deterrent;  the  concept  is  sound.  Top  management  leaders  of 


SAF/AQXD  should: 

•  Continue  to  support  the  principles  of  GPRA, 

•  Use  the  suggested  goals  and  performance  measures  as  a  starting  point, 

•  Advocate  recurring  strategic  team  meetings  to  refine  and  analyze  SAF/AQXD’ s  strategic  and 
performance  plans,  subsequent  measures  and  results, 

•  Construct  an  avenue  to  solicit,  reflect  and  act  upon  customer  (students,  supervisors, 
functional  boards,  consortium  schools)  feedback, 

•  Investigate  the  integrity  of  performance  data,  and 

•  Maintain  the  aggressive  schedule  to  integrate  data  systems. 

Even  though  GPRA  has  been  in  existence  since  1993,  it  is  still  considered  to  be  in  its 
infancy,  slowly  overcoming  the  challenges  plaguing  agencies.  Movement  to  performance  based 
management  does  not  happen  overnight.  However,  with  time  and  perseverance,  SAF/AQXD  can 
master  goal  setting  and  performance  monitoring  and  will  reap  the  benefits  of  performance  based 
management. 

Conclusions 


The  acquisition  workforce  must  be  prepared  to  face  the  challenges  of  the  21st  century.  To 
face  these  challenges,  we  must  ensure  that  our  acquisition  workforce  is  well  trained,  flexible  and 
more  effective  than  ever  before.  The  basic  concepts  of  the  GPRA  have  been  addressed  so  that 
we  may  gain  knowledge  of  its  potential.  Understanding  the  results  of  our  performance  will 
ensure  the  protection  of  our  most  critical  resource,  the  acquisition  workforce.  They  are  the  ones 
to  get  the  product  or  the  service  to  the  warfighter,  faster,  better  and  cheaper. 
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SAF/AQXD  is  clearly  on  their  way  to  mastering  goal  setting  and  performance  monitoring. 
The  principles  of  GPRA  can  be  incorporated  to  improve  the  primary  business  of  the 
organization — effective  management  of  APDP.  SAF/AQXD  has  developed  a  strategic  roadmap. 
At  the  forefront  of  their  strategic  planning  is  top-level  management  support  and  lower  level 
enthusiasm  to  exploit  strategic  opportunities,  like  the  concept  of  GPRA.  They  have  even 
recognized  and  are  taking  strides  in  integrating  their  data  collection  systems.  The  information  in 
this  research  paper  serves  as  a  guide  in  implementing  the  principles  and  the  proposed  goals  and 
measures  and  recommendations  serve  as  stepping  stones  in  implementing  the  concept. 
SAF/AQXD  has  all  the  ingredients  to  make  this  effort  a  success,  to  protect  its  resources  and  be 
available  to  respond  to  all  their  customers,  including  the  public. 


Notes 

1  Steven  Watkins,  “Performance  Based  Government  Sparks  Concerns,”  Federal  Times  35, 
no.  25  (26  July  1999):  3. 
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